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Abstract: In this paper the design of a workflow support tool for competence based distance learning in 
a group setting is discussed. The design is based on a stakeholder analysis and crash-tested in an 
actual course setting. Preliminary findings suggest that some well-known problems have been solved, 
but further more in depth research is needed to assess the quality of the design with respect to more 
subtle issues. 
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1. Introduction 

Recently, the Open University in the 
Netherlands (=0U) has endorsed 
competence based learning as the future 
didactical framework for course 
development and exploitation. The Open 
University is an institute, which provides 
distance-learning courses. Typically, the 
students have already some working 
experience and are eager to develop their 
careers. Students are increasingly critical 
with respect to the quality of the education 
they get from the OU. In addition, 
government funding is under constant 
pressure, which limits the resources 
available for course development and 
exploitation. For training students in 
business process engineering, the OU 
uses competence based distance learning 
in a group setting (GCBDL Courses). 
Typically, business process engineers 
operate in multi-disciplinary groups, and 
the success of a business-reengineering 
project very much depends on 
collaboration and communication skills of 
the group participants. In addition, 
collaborative learning stimulates the 
communal sense with students (Seufert, 
2002). Research by Kear and Heap 
(Kear&Heap, 1999) suggests that although 
students complain about the additional 
overhead and limited freedom of 
collaborative learning, the majority of the 
students believe universities should 
continue in providing collaborative learning 
courses. Thus, academic discussion and 
interchange of ideas is stimulated 
providing a noticeable added value to the 
participants. The Internet as a medium for 
communication within distance learning 
has opened new possibilities of distance 
learning in general. 


Competence based learning has received 
much attention from a didactical 
perspective, in which predominantly the 
effectiveness of the learning processes of 
the students is addressed. In addition, 
much attention has been paid to the 
development of electronic learning 
environments, in which the presentation of 
course content and rather crude 
communication facilities was paramount 
(e.g. see Martin, 2003). Unfortunately, 
almost no attention has been paid to the 
consequences of competence based 
learning on a large scale and on a routine 
basis. Some courses at the Faculty of 
Management Science at the OU already 
follow the pattern of competence based 
learning, and first informal experiences 
indicate that, in particular during the 
exploitation stage, amongst other issues, 
the administrative workload for students 
and supervisors can be prohibitive and 
that supervisors and students sometimes 
lack control to finish their assignments on 
time. 

In this paper, we will focus on the 
implementation aspects of this type of 
GCBDL courses on a routine basis. We 
will analyse systematically the workflow 
requirements, which are associated with 
this type of course from an industrial 
engineering perspective. This analysis will 
serve as a basis for improvement of the 
workflow. In particular, we will concentrate 
on the potential for improvement of 
GCBDL-supporting information systems. 
We will use the business-reengineering 
course as a test case. This course is 
offered to about 100 students per year, 
who start in batches of about 15 to 20 
students from several different master 
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programs at various moments throughout 
the year. In addition, some course material 
is used in a similar setting for company 
training purposes. External supervisors 
are contracted as needed. 

2. Problem definition 

The competence based learning strategy 
leads to courses where students are put in 
a realistic situation in which they have to 
demonstrate if and how they solve certain 
problems. Absorption of cognitive 
knowledge is considered insufficient. The 
key objective is to be able to use 
knowledge to solve realistic problems. In 
practice, this learning strategy can be 
translated into course patterns, which 
make use of case descriptions and 
assignments for groups of students. On an 
academic level, the assignments typically 
require the students to make an analysis 
and construct a systematic and clear 
argument for a solution. Compared to 
traditional cognitive based learning 
approaches, such assignments will require 
manual feedback from scarcely available 
professional supervisors. In addition, 
supervisors have to keep track of the 
students’ states and the responses as the 
“body of evidence” for the students’ 
assessment. Although no formal in-depth 
studies have been discovered in which 
traditional courses are compared with 
similar scoped GCBDL-courses, 
experienced supervisors estimate the 
extra effort roughly to about 2 to 3 times 
compared to a traditional course design. 
Veen et al (Veen,. 1999) reports similar 
findings on a workload increase with 
GCBDL-courses. 

In general, regular course evaluations 
suggest that many students perceive that 
GCBDL-courses increase the workload. It 
is no longer sufficient to prepare for a 
single examination, but instead, the 
students have to produce a full final report, 
which they all have to agree upon, and 
possibly several written responses to 
intermediate assignments. All these 
symptoms, may be acceptable if 
competence based learning is used 
incidentally. However, if used on a routine 
basis some further research seems 
justifiable. 

Our basic research question 
boils down to: “How can we 
implement GCBDL courses 
efficiently?”. 


2.1 Methodology 

The symptoms identified in the previous 
section indicate that we need to analyse 
the workflow of GCBDL courses to 
understand its management and 
administration problems better in order to 
be able to facilitate a GCBDL-workflow. 
We will follow the design cycle as the 
primary guideline in our approach. The 
design cycle entails the following stages: 

■ Determine design objectives; In this 
first stage we need to determine what 
criteria are relevant and must be met 
for an acceptable workflow. Besides 
the course design criteria we need to 
determine what stakeholders exist and 
what their requirements actually are. 

* Considerations for design; In this 
stage, the requirements from the 
previous stage have to be interpreted 
and related to the main options for 
facilitating the workflow. Finally, the 
most suitable options will be chosen for 
the development of a prototype 
instrument. 

■ Develop a prototype; A full prototype 
version of the workflow support 
instrument will be developed and 
discussed 

* Evaluate prototype; Eventually, a 
prototype needs testing. In this stage a 
working version of the workflow 
instrument will be put to the test with a 
real life course 

* Re-iterate if necessary; In case the 
evaluation reveals shortcomings in the 
prototype, the design cycle can be re¬ 
initiated at each previous design stage 

2.2 The design of a workflow 

2.2 .1 Stakeholder analysis 

For a successful design of a workflow 
support system it is essential to have a 
clear picture of the relevant stakeholders, 
or actors in this case and what their 
interests in such a workflow support 
system would be. In the following sections 
we will discuss the stakeholders and their 
interests in depth. The insights presented 
here originate from regular evaluations 
that are carried out after each course run 
of the Business process engineering 
course and regular evaluation meetings 
with supervisors and examiners of this 
course. 
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Developer 

The didactical model, used or designed by 
the developer, is leading for all other roles. 
The developer is more or less a generic 
role, he may comprise a team with mixed 
specialisations in the content area, 
pedagogy, programming, legislation, 
finance, etc.. Course development may 
require some attention to project 
management. In particular, if the course 
material is novel and not much existing 
content can be reused. Besides, 
achievement of the quality standards for 
educational content demanded by the 
institution offering the course and the 
accreditation boards, other secondary 
requirements must be met by the 
developer. Depending on the situation at 
hand, a wide variety of restrictions must be 
taken into consideration. Implementation 
of a course in a routine workflow is 
considered an administrative duty, for 
which any effort should be minimised. 

Examiner 

The examiner, being responsible for the 
examination of students, is an officially 
appointed expert on the course material at 
hand. He assesses the performance from 
the student using the standards the 
developer of the course has set. The 
outcome of such an assessment is usually 
summarised in a number from a preset 
range. The standards may encompass 
many heterogeneous aspects such as the 
level of knowledge that has been 
accumulated, the throughput time to 
complete the course, the amount of 
supervision that was needed to guide a 
student towards the completion of a 
course, etc. Much of the feedback of 
intermediate course assignments can be 
delegated to a so-called supervisor role. In 
that case the examiner requires that 
supervisors adhere to his standards for 
providing feedback in terms of timing, 
content and student progress in general. 
The examiner rating has a legal status, i.e. 
disputes between students and examiners 
may be taken to court if conventional 
arbitration fails. This emphasises the 
administrative precision and formalism the 
examiner has to practice. 

The examiner must collect all evidence in 
support of his assessment and he must 
assure that the course has been followed 
in conformance with the prescribed 
didactical model. 
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Student 

The student is the main beneficiary of a 
course. He executes the assignments and 
tests prescribed by the examination 
standards in accordance with the 
didactical model. The student’s main 
motivation comprises two elements. 

* a gain in competences on the particular 
area the course within the program he 
selected and; 

■ the certificate he receives upon 
successful completion, which in turn, 
depends on a positive rating by the 
examiner. 

Ratings and comments from a supervisor 
or the examiner must be clearly linked to 
student responses to the required 
assignments. 

Secondary, but nevertheless, important 
motivational aspects are the facilities that 
are provided in order to help him to 
complete a course successfully. Typically, 
in GCDBL-courses additional effort is 
needed to manage the group process. If a 
GCDBL-course is large and involves a 
substantial number of assignments over 
an extended period of time, project 
management type of control of the group 
progress may be welcomed. 

Manager 

The manager is responsible for the overall 
economic efficiency and the market 
validity of the courses offered to the 
students, a.k.a. the clients of the 
educational institution. The manager 
makes the decisions to introduce a course 
or a program to a certain market and what 
features are needed for a successful 
exploitation, the development cost that can 
be justified and the management that is 
required for the development, promotion, 
maintenance and exploitation of a course. 
Given the wide range of responsibilities, 
this role is seldom attributed to a single 
person within an organisation. In the end, 
the dean is formally responsible for all 
aspects mentioned here, but ideally, all 
individuals involved in the processes 
mentioned above will carry this 
responsibility more or less implicitly. 

The manager role is interested in the 
overall efficiency of course runs per 
course. The effort spent by the 
supervisors, examiners, and students is 
monitored and should not exceed a preset 
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budget. Additional quality requirements 
such as the percentage of students who 
completed a course successfully and 
qualitative student evaluations 
complement the manager’s information 
requirements. The manager has no direct 
involvement with an individual course run. 
Due to budget pressure this role has 
gained in importance significantly and will 
have a much larger impact on a work flow 
system than a few years ago. 

Supervisor 

The supervisor acts as an intermediate 
between the student and the examiner. 
The supervisor assists the examiner in 
collecting “evidence” of the student’s 
performance during a course run. In 
addition, he is the first member of staff a 
student contacts in case he requires 
assistance, advice and feedback on his 
work. The supervisor is also responsible 
for the progress students are making. To 
do this, the supervisor needs to know the 
actual planning of the students at any 
time. Depending on the course 
requirements and the sophistication of the 
course material, a supervisor may not 
need to have full expertise on the content 
subjects of the course at hand. In such 
cases, a supervisor’s responsibility is to 
stimulate the student’s pace and to 
provide only basic safeguards from a 
content perspective. In other words, the 
supervisor resembles more a principal 
than an instructor. 

Administrator 

The administrator is responsible for the 
official admission of students to participate 
in a course. He may check the student’s 
admission qualifications before 
commencing with the actual enrolment of 
students. At the end of a course, an 
administrator may want to update the 
official student dossiers with the 
examiners rating and file any other 
required documents. 

Veen et al. (Veen, 2001, and also 
Veen,Collis,Diepen&Andernach, 1997) 
have studied the management challenges 
introduced with collaborative learning 
extensively. His analysis can be 
summarised as twelve major problems of 
group workflow problems: 

Problem planning, operationalisation & 
monitoring 


Table 1: (Source: Veen, 2001, p.45) 


1 

Groups do not have a clear picture of 
what is expected of them, planning 

2 

Groups have problems with planning and 
procrastination, planning 

3 

Groups have problems with organizing 
work between meetings. 

operationalisation & monitoring 

4 

Groups have problems with access to 
deliverables and comments. 

operationalisation 

5 

Group members do not take a fair share 
of the work, operationalisation & 
monitoring 

6 

Instructors lack overview of the progress 
of groups, monitoring 

7 

Different instructors treat groups in 
different ways, operationalisation & 
monitoring 

8 

Instructors have difficulties to continue 
their work at a distance. 

operationalisation 

9 

Students have limited awareness of other 
group members, operationalisation & 
monitoring 

10 

Conflicts arise due to poor 

communication, operationalisation 

11 

Students do not start using telematic 
support tools, operationalisation 

12 

Groups have to wait too long for 
instructor and peer comments 

operationalisation & monitoring 


A workflow support system should address 
the problems identified above. 


2.2.2 Design considerations 

To address the requirements of 
stakeholders a number of facilities should 
be provided by a workflow support system. 
In the following section the most important 
design directions will be discussed. 

Providing a structured path 

Examining the way groups worked in an 
earlier version of the Business¬ 
reengineering course, which was not 
supported with workflow support tools, we 
have recognized a pattern described by 
McConnell (2002). 

a) A long first phase characterized by 
considerable negotiation between the 
members of the group working closely 
together. Several sub-phases are 
evident in the groups’ work. 

b) A medium-length second phase 
characterized by the group organizing 
itself and busying themselves with 
particular parts of the research around 
the particular problem. 
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c) A short third and final phase, 
characterized by production. 

Before students are asked to work in 
groups they will meet at least once at the 
beginning of a course run. We assume 
that no real substitute exists for getting 
acquainted quickly. No tool is provided for 
this stage of a group project. Also, at this 
stage it should be agreed upon who is 
responsible for what aspects in the 
workflow. In particular, the problems 9 and 
1 in Veen’s list are addressed. 

All assignment responses and comments 
should be archived, logged and made 
visible to the individual group members, 
supervisor and examiner, including all 
revisions of student responses and 
supervisor comments. 

The supervisor role should have final 
control on when an intermediate 
assignment has been answered 
satisfactory and a group can move on to 
the next assignment (problems 4 and 6). 

Monitoring deadlines 

It is a group responsibility to set and 
monitor delivery dates for sending 
responses to (intermediate) assignments. 
Equally, supervisors have to answer within 
a preset time frame Setting these time 
standards would allow an automatic alarm 
system sending appropriate warnings to all 
participants, who have trouble in meeting 
the deadlines. In addition, all assignment 
related communication events should be 
logged. Thus, all essential data is 
available in a formal and relatively 
indisputable way, which should minimise 
defensive behaviour of all roles involved 
(problem 6 and 12). 

Maximum accessibility 

Any support tool can be easily bypassed if 
users disagree with the way in which the 
tools more or less enforce a certain 
standard workflow. If this happens, the 
workflow itself is in jeopardy. In general, 
we assume that examiners and 
supervisors are bound to use the support 
tool and must be intolerant to any 
deviation. On the other hand, students 
may have very good reasons to reject the 
facilities provided to them. The tool may 
be too complex and requires a steep 


Harry H. Martin and Eric H. Willems 


learning curve, or the technical 
requirements of the tool exceed the 
available PC specifications, to name just 
two of the more frequent reasons for 
rejection (problems 11 and 4). 

A possible solution to this problem may be 
to provide the tool as a web-application 
which can be accessed by most browsers 
on any internet enabled PC anywhere in 
the world at all times. Secondly, the design 
objective could be to minimise the 
functions to those that are absolutely 
necessary and avoid a “Jack of all trades” 
approach (“Simplicity versus functionality”, 
see Veen,Collis&Jones 2001). The 
philosophy is that, the fewer functions are 
provided, the fewer reasons are available 
to bypass the tool. We expect that other 
tools available in the market, which the 
students can choose to their liking, can 
easily add missing non-critical 
functionality. In this setting, self-controlled 
selection of facilities provides the students 
with a certain degree of freedom to choose 
their own instruments and should actually 
encourage students to take initiatives and 
go ahead. 

Simplicity as a design paradigm is also 
applied in the level of sophistication a 
workflow support system should have. We 
were looking for simple and robust 
solutions. E.g., instead of trying to design 
an advanced automated agent system that 
could act as the primary supervisor (see 
Palazzo et all, 1998), we concentrated on 
just making all relevant data easily 
accessible to a supervisor. 

2.2.3 The basic workflow 
implementation 

A workflow support tool, named the “Back 
Office” was developed to implement the 
requirements listed in the previous section. 
The Back Office was developed with 
Delphi 7 Enterprise Edition 1 , Intraweb 
version 5" and Firebird DBMS version 
1.03'". We have separated the workflow in 
roles. Each role has its own 
responsibilities and tasks in the overall 
workflow. For each role a separate web- 
application has been developed and 
installed on standard PC with permanent 
Internet connection. 
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Table 2: Key functions of the back office application 


Type of 
application 

Role 

Key functions 

W 

Examiner 

annex 

Course 

developer 

Define/maintain course structure, assignments (group and individual) 
Define/maintain FAQ for supervisors to aid them in providing feedback 
to student work 

Compose/change student groups. 

Monitor individual and group process 

Ability to email all or selection of all current participants 

Manage course runs (timing) and authorise supervisors and enrollers 

Rate students 

Provide digital course material (for download only by students) 

Export course run data to XML-file for offline analysis 

W 

Enroller annex 
Administrator 

Enlist students 

Authorise/(un)block/withdraw student participation 

Welcoming of new students via automated email. 

Monitor final rating from examiner 

Authorise examiner for closing down a course run. 

W 

Supervisor 

Monitor progress of individuals and groups 

provide and distribute feedback to students/groups (times and contents 
are logged) 

Facilitation of automated email to individuals and groups 

Rating of assignments 

Decide whether individuals/groups are allowed to proceed to the next 
assignment or to improve their current work. 

W 

Student 

Define/maintain individual and group planning (via setting deadlines for 
each assignment) 

Download course material and standard comments provided by 
examiner 

Read and respond to supervisor feedback 

Maintain a shared group file repository 

Ability to send automated emails to supervisor or group members. 

W 

Due date 

monitor 

Send automated emails to all roles who have exceeded certain 
deadlines: 

Students approaching/hitting/exceeding deadlines (warning) 

Supervisors approaching/hitting/exceeding feedback deadlines 
(warning) 

Minimum planning resolution defaults to one day, but can be changed. 

W= web based, A= automatic, runs once per night. 

Email is sent by the web server, instead by a local email client and is referred to as “automated email”. 


In parallel a monitor program runs daily to 
check if additional warning messages 
should be send and to perform automatic 
backups of the database. This monitor 
program introduces a “push element” into 
the back office, whereas traditional web 
applications are “pull-driven”. An action of 
a student or a supervisor always calls for 
another action. It is hoped that, in doing 
so, the likelihood that the group progress 
stalls is lessened significantly (see also 
Hauswirth 1999). The separate 
applications and their key functions are 
summarised in table 2. 

Implementing the back office workflow. 

To enable testing the workflow, a web- 
application and central database have 
been developed. All web pages resemble 
normal windows data entry forms. 
However, these forms are generated on a 
central server and displayed within a 


browser on a local PC. All persistent data 
has been modelled into a database 
management system accessed by the 
web-application. This implementation 
provides instant database update 
capabilities and access from users on any 
PC connected to the Internet. Access to a 
web-application is achieved with a login- 
procedure in order to protect the users’ 
privacy. 

In addition to the support of the basic 
workflow several smaller enhancements 
were implemented to encourage users to 
use the back office. E.g. student groups 
have a shared file repository they can use 
privately. Scheduled assignments are 
presented graphically (see figure 1). 
Special attention has been paid to the help 
system for the student web application. 
Help is available in full text and so called 
Flash movies. 
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One of the design considerations was 
transparency of data. In the 
implementation, this aspect has been 
translated into extended logging of all 
relevant actions by any user and full 
visibility of the data that is related to a 
logged action. E.g. date and time are 
recorded of any assignment related 
response from a student, which can be a 
typed response in the browser itself or via 


an upload of files. In turn, all supervisor 
comments are logged as well. In addition, 
all uploaded files, comments, etc. are 
visible and downloadable (see figure 2 for 
an example). According to Armatas 
(Armatas c.s., 2003) distance students 
value having full access to their own data 
and being in control of their schedule 
much more than on campus students do. 



Figure 1 : A graphical presentation of a student schedule. 
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3 
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Figure 2: An example of a table showing all student assignment responses and comments 
from the supervisor. 
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3. Conclusion and future 
challenges 

At the time of this writing the prototype 
workflow support tool (the Back Office) 
has been tested for stability with a number 
of groups of business reengineering 
students. So far, 5 groups with in all 20 
students and two supervisors have used 
the Back Office on a regular basis for a 
period of about half a year. Students and 
supervisors were asked to report any 
dissatisfaction immediately to the research 
team. 

So far, no in depth evaluation of this 
prototype and the considerations behind 
this software has been carried out yet. 
Therefore, no solid conclusions can be 
drawn at this time. However, no major 
complaints have been received from the 
students and the supervisors. Minor issues 
such as long upload times could usually 
be attributed to large bitmap files students 
have included with their responses. Once 
the students were made aware of this, 
further problems could be avoided. 
Remarkably few requests for support from 
students or supervisors were made. This 
may be an indication that the minimalist 
approach does work as desired. No 
students have rejected the tool. 

The Back Office has solved some of the 
twelve problems identified by Veen (Veen, 
J. van der, 2001) right away. Groups have 
direct access to deliverables and 
comments (problem 4). This may not be a 
attributed to just appropriate tool design, 
but also to the quality of internet 
connections students have nowadays. 
However, a potential threat for any Internet 
based support tool is the Firewall policy of 
Internet providers. Although no real 
problems emerged during the first 
debugging tests, one can expect that 
different Firewall policies may seriously 
impair legitimate Internet communications. 

Different instructors can now treat groups 
much in the same way and use the same 
criteria to support their feedback to the 
groups (problem 7). Further studies may 
be needed to discover if standard 
feedback, or FAQ lists, is the best solution 
indeed. Groups don’t have to wait too long 
for comments (problem 12). Automatic 
monitoring provides warning signals to 
those who are late with their task. 


The problem of students not using 
telematic support tools (problem 11) and 
poor communication (problem 10) may not 
be as dramatic nowadays due 
technological advancements in 
communication technology and the 
widespread acceptance of these 
technologies by the students. At this 
stage, no complaints from students have 
been received, indicating that standard 
communication means were insufficient, or 
that the quality of student responses was 
hampered due to insufficient 
communication. 

The remaining problems may be 
addressed partially by the Back Office and 
partially by the way in which a workflow is 
organised. The contribution of the Back 
Office in these problem areas does require 
further investigation. E.g. to assess 
whether group members do not take a fair 
share of work (problem 5) would require 
more insight on how groups actually 
allocate the tasks at hand. Unlike in 
classroom teaching, in which a supervisor 
is a direct witness of actual group 
processes, in distance learning a group 
process is largely hidden from the 
supervisors view. To ensure a sound 
group participation, different strategies can 
be followed. Daradoumis et all 
(Daradoumis, 2002) log all student events 
in their system, forcing students to route 
all of their communication to the official 
channel to achieve recognition (formative 
tracking). Such a system would require 
almost 100% uptime and total ease of use 
for the users to make this a serious option. 
A totally different approach is suggested 
by Seufert (Seufert 2002). He argues that 
if the learning environment and the 
working environment is connected, 
students will be more motivated as 
learning success can be transferred into 
their work and vice versa. However, this 
may be difficult to realise in practice. Vick 
& Johnson (Vick&Johnson 2005) solve this 
problem by providing specific software, 
which supports brainstorm and problem 
solving software for geographically 
dispersed group members. The software 
requires that all group members contribute 
evenly and synchronously. In a sense, a 
live guided discussion between group 
members is facilitated and logged for 
future references. So far, the problem has 
not been tackled with the aid of some back 
office functionality, but through course 
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design. Currently, for the Business 
reengineering course a strict pattern of 
responsibilities is agreed upon during a 
kick-off meeting at the beginning of the 
course, but this pattern is not strictly 
enforced. In addition, each individual 
student is required to send a personal 
reflection report on the group process and 
his personal perspective of his 
contribution. 

The discussion presented above makes 
clear that the “Free-riders” problem is 
complex and needs further investigation. 

Currently, the workflow doesn’t support 
groups or individuals within groups to 
interact with each other as a community. 
Lou (Lou, 2004) suggests that 
communities provide an even richer 
learning environment to students with 
unique opportunities to implement peer 
assessments. Further, long term research 
in this area is needed to explore the 
possibility to enhance our Back Office 
design with community building. 

Implementations of group management 
software at the Fernuniversitat Hagen 
(Haake et all, 2002) suggest that in the 
area of enrolment and group forming 
improvements can be made. In their 
implementation, individual students select 
the groups they want to collaborate with 
themselves. Initially, an individual in a 
student pool solicits for a group 
membership of a group of his preference. 
The existing group members have to 
accept, or reject the new applicant before 
moving on. This setting demonstrates 
three interesting benefits. Since students 
group themselves, essentially no 
significant support from the university staff 
is needed, saving valuable resources. In 
addition, all responsibility for the success 
of a group now rests solely with the 
individual group members. The university 
staff cannot be blamed for a failed group 
process. Also, batching students in groups 
at regular intervals introduces peaks in the 
workload of examiners and supervisors. In 
the pooling system, there is no need to 
batch group formation. As soon as there 
are enough students available, students 
can start grouping. Further assessment of 
the “Hagen” system is needed to discover 
if this method can be applied in a back 
office system at the OU as well. 


Harry H. Martin and Eric H. Willems 
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